At present, according to the standards intended for low bit rates for the purpose of image communication, such as H.263 and MPEG4, the motion compensation is performed to compress data by employing a correlation between frames.
A motion of a moving picture is represented by successive frames which are recorded at certain time intervals. Here, descriptions will be given of two frames which are adjacent to each other in the time axis direction (i.e., a present frame and a previous frame that is temporally antecedent to the present frame).
When the present frame (hereinafter, referred to as a processing target frame) is to be compressed, it is common to divide the frame into plural blocks and then carry out the compression processing in units of these blocks (hereinafter, referred to as processing target blocks). A motion vector search technique of predicting the processing target block on the basis of a similar block (hereinafter, referred to as a reference block) in the previous frame (hereinafter, referred to as a reference frame) to increase the compression efficiency is widely known.
Here, the motion vector search technique will be described in more detail with reference to FIG. 1.
In the compression process, a block in a reference frame 103, which is similar to a processing target block 102 in a processing target frame 101 is searched as shown in FIG. 1. At this time, a range to be searched is limited within a reference area 104. The reference area 104 is allowed to extend to an external area of the reference frame 103. In this case, an expansion processing is carried out in accordance with pixel expansion patterns 1 to 9 as shown in FIG. 4. For example, when the reference area corresponds to pixel expansion pattern 7, pixels in a reference frame 401 are expanded according to a predetermined rule toward the outside of the reference frame as shown in an example 411 of area expansion toward outside the frame. The area expansion processing is a technique that was applied first to the standards intended for low bit rates such as H.263 and MPEG4.
Next, the motion vector will be described with reference to FIG. 2.
Initially, as described above, a block comprising NX pixels×NY pixels within a reference area 202 comprising MX pixels×MY pixels, which has the highest degree of similarity with a processing target block 201 comprising NX pixels×NY pixels is searched. It is assumed here that the processing target block 201 and a reference block 205 have the highest degree of similarity. In this case, a motion vector 204 is represented as (MVx, MVy). Then, the motion compensation is performed on the basis of the motion vector 204. The reference block 205 may comprise not only pixels at integral positions but also pixels at positions of numbers including decimal fractions. In such case, each of MVx and MVy as the factors of the motion vector 204 has naturally a decimal fraction.
The motion vector search is performed as described above, and consequently the direction and the distance by which the processing target block is to be moved, i.e., motion compensated are shown by the motion vector, thereby performing an encoding process for the moving picture. In a decoding process for the moving picture, the motion vector search is not required and only the motion compensation is performed.
However, in this area expansion processing, it should be decided whether each pixel is within or outside the reference frame area, and further expansion of pixels according to the predetermined rule should performed, whereby the operation loads are unfavorably greatly increased.
To solve this problem, “Image processing method and image processor” are disclosed in International Application No. PCT/JP99/06997, which will be described with referring to FIG. 10.
FIG. 10 is a block diagram illustrating a structure of the image processor that is described in this application. As shown in FIG. 10, the image processor includes an image input/output unit 10 for receiving inputted image or outputting images to be displayed; a DMA bus 11 for performing data transfer between the image input/output unit 10 or a coding/decoding unit 13 and an external memory 12; an external memory 12 for storing image data or coded data; and a coding/decoding unit 13 for carrying out a coding/decoding process, with using an effective image data area (area within a reference frame) and an expanded area.
The coding/decoding unit 13 includes a data processing unit 14 for coding/decoding image data; a memory 15 for storing the image data read from the external memory 12; a control unit 16 for outputting contents to be processed and a processing timing to the data processing unit 14, and outputting a start address of an area corresponding to data which are transferred from the memory 15 to the data processing unit 14, to an address generation unit 17; and an address generation unit 17 for controlling the data transfer from the memory 15 to the data processing unit 14.
The address generation unit 17 comprises a two-dimensional address generation unit 18 for generating an address on the basis of setting information from the control unit 16, and an address conversion unit 19 for converting the address generated by the two-dimensional address generation unit 18 into an access address in the memory 15.
The operation of the image processor that is constructed as described above will be described.
An image inputted to the image input/output unit 10 is initially subjected to resolution conversion by the image input/output unit 10 to be converted into an image size corresponding to a coding target, and stored in the external memory 12 via the DAM bus 11.
On the other hand, the coding/decoding unit 13 decides a rectangular area to be read, i.e., an access rectangular area in an expanded logic space which is obtained by adding an expansion area to an effective image data area, and stores image data corresponding to the effective image data area stored in the external memory 12 within the access rectangular area, in the memory 15.
When a coding target image or a decoding target image is transferred from the memory 15 to the data processing unit 14, the control unit 16 decides a data processing rectangular area to be transferred to the data processing unit 14 in the access rectangular area, and sets the setting information for enabling the address generation unit 17 to generate an access address corresponding to the data processing rectangular area.
The address generation unit 17 successively generates an address of the data processing rectangular area on the basis of the setting information. Then, the address conversion unit 19 decides a pixel expansion pattern corresponding to the address generated by the two-dimensional address generation unit 18 by employing a table comprising frames boundary positions in a reference area which are patternized by applying the nine pixel expansion patterns (i.e., pixel expansion patterns 1 to 9) shown in FIG. 4, and converts the generated address into an address within the effective image data area when the address is outside the effective image data area.
The stored data are read from the memory 15 in accordance with the address generated by the address generation unit 17, and outputted to the data processing unit 14.
As described above, it is suggested that the nine pixel expansion patterns (pixel expansion patterns 1 to 9) as shown in FIG. 4 are applied to patternize the frame boundary positions in the reference area, whereby the area expansion processing can be implemented by a relatively small circuit.
However, the area expansion apparatus and area expansion method are inventions which suppose applications requiring both of the coding and decoding processes for moving pictures, including the motion vector search process for deciding a motion vector, for example like videophones, and do not have a suitable structure for applications such as TV broadcasting for which only the decoding process for moving pictures is satisfactory. More specifically, this apparatus is required to have a memory capacity that is necessary for the motion vector search in the image coding process, whereby this structure is over-specification in the applications that requires only the decoding process.
Specific descriptions will be given of a case where fcode=1 in MPEG4. As shown in FIG. 3, a reference area 301 comprising 48 pixels×48 pixels is represented on coordinates with a position of pixel P302 as an origin. Though not shown, a reference block comprises 16 pixels×16 pixels. In this case, since a half-PEL precision search is allowed in MPEG4, data corresponding to a rectangular area comprising at least 17 pixels×17 pixels within the reference area 301 are sufficient to generate the reference block comprising 16 pixels×16 pixels. In the decoding process, it is satisfactory that the apparatus includes a memory that can store 17 pixels×17 pixels. However, the above-mentioned method and apparatus require a memory that can store 48 pixels×48 pixels which are required for the motion vector search in the coding process. More specifically, as shown in FIG. 5, the image decoding process only requires pixels in a reference block 502, while the conventional area expansion apparatus requires a memory having a capacity that can store 2304 pixels in a reference area 501.
Now, it is assumed that data in a reference area including data of a rectangular area comprising 17 pixels×17 pixels are simply subjected to the area expansion processing. In the case of the pixel expansion processing that is disclosed in the above-mentioned International Application No. PCT/JP99/06997, focusing attention to the pixel expansion patterns as shown in FIG. 4, there are only nine patterns of boundaries. Further, the frame boundary is defined by a combination of four demarcation lines. Therefore, the boundary decision conditions and pixel expansion methods corresponding to these conditions are made associated with each other in one-to-one relation. These associated patterns can be easily mounted on a small-scaled circuit.
When the expansion processing is to be implemented within a rectangular area comprising 17 pixels×17 pixels, the reference block 502 may be located at any position in the reference area 501 as shown in FIG. 5. Further, the frame boundary may be located at any position in the rectangular area comprising 17 pixels×17 pixels. Therefore, it is difficult to make the boundary decision conditions and the pixel expansion methods corresponding to the conditions correspond in one-to-one relation, and the boundary decision and the calculation of addresses corresponding to pixels to be expanded must be performed in units of pixels. That is, the calculation for expanding pixels as shown in the example 411 of area expansion toward outside the frame in FIG. 4 must be performed, resulting in the complexity in the expansion processing. As described above, as the frame boundary is located at any position in the rectangular area comprising 17 pixels×17 pixels, the boundary decision and the pixel expansion according to the predetermined rule become complicated.